On Low-Complexity Approximation of Matrices

The operation ‘multiplication of a vector by a matrix’ can be represented by a computational scheme (or model) that acts on the entries of the vector sequentially. The number of intermediate quantities (‘states’) that are needed in the computations is a measure of the complexity of the model. If that complexity is low, then not only multiplication, but also other operations such as inversion, can be carried out efficiently using the model rather than the original matrix. In the introductory sections, we describe an algorithm to derive a computational model of minimal complexity that gives an exact representation of an arbitrary upper triangular matrix. The main result of the paper is an algorithm for computing an approximating matrix with a model of (much) lower complexity than the original — as low as possible for a given tolerance on the approximation error. As measure for the tolerance we will use a strong norm which we will call the Hankel norm. It is a generalization of the Hankel norm which is used in the classical model approximation theory for complex analytical functions.